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Following a pretest, 1 1 participants who were naive with regard to various algebraic and 
trigonometric transformations received an introductory lecture regarding the fundamentals of 
the rectangular coordinate system. Following the lecture, they took part in a computer- 
interactive matching-to-sample procedure in which they received training on particular formula- 
to-formula and formula-to-graph relations as these formulas pertain to reflections and vertical 
and horizontal shifts. In training A-B, standard formulas served as samples and factored formulas 
served as comparisons. In training B-C, factored formulas served as samples and graphs served as 
comparisons. Subsequently, the program assessed for mutually entailed B-A and C-B relations as 
well as combinatorially entailed C-A and A-C relations. After all participants demonstrated 
mutual entailment and combinatorial entailment, we employed a test of novel relations to assess 
40 different and complex variations of the original training formulas and their respective graphs. 
Six of 1 0 participants who completed training demonstrated perfect or near-perfect performance 
in identifying novel formula-to-graph relations. Three of the 4 participants who made more than 
three incorrect responses during the assessment of novel relations showed some commonality 
among their error patterns. Derived transfer of stimulus control using mathematical relations is 
discussed. 

DESCRIPTORS: mutual entailment, combinatorial entailment, mathematical relations, 
stimulus equivalence, novel relations, matching to sample, relational frame theory 


Mathematicians have often assumed that 
their students can, or should, be able to relate 
various formulas to their graphical representa- 
tions rather spontaneously. For example, one 
commonly used precalculus textbook advises 
the reader that particular types of formulas are 
critical to the understanding of all mathematical 
relations and states, “Whenever you encounter 
one of them, you should see a mental picture of 
its graph” (Sullivan & Sullivan, 1996, p. 114). 

Nevertheless, a wide range of mathematical 
concepts remains elusive for large segments of 
our population, and a lack of mathematical 
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preparation handicaps students who attempt 
postsecondary education and cannot compete 
in traditional math courses. Seventy-five percent 
of community college students and 50% of 
university freshmen lack the requisite skills to 
succeed in traditional college-level math courses 
(Burley, 1997; Center for Science, Mathematics, 
and Engineering Education, 1996; “CUNY 
Trustees,” 1999; “Educational Catch-Up,” 
1998; Ravitch, 1997). In addition, 50% of 
all college students fail developmental course 
offerings (EnabLearning, 2002). In Texas, 70% 
of community and technical college students 
fail the statewide assessment exam and need 
developmental courses (Texas Higher Educa- 
tion Coordinating Board, 1999). 

Until recently, the analysis of problem 
solving and similar cognitive phenomena has 
been almost exclusively the domain of cognitive 
psychology (cf. Reese, 1992). Particularly, as 
these issues pertain to the exploration and 
understanding of mathematical concepts (e.g., 
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Anderson, Corbett, Koedinger, & Pelletier, 
1995) or related problem-solving and analogical 
concepts, cognitive psychology (e.g., Vosniadou 
& Ortony, 1989) has sustained a long and 
vigorous applied research agenda (e.g., 
Vygotsky, 1978). The study of equivalence 
relations (e.g.. Lane, Clow, Innis, & Critchfield, 
1998) and relational frame theory (RFT; Hayes, 
Barnes-Holmes, & Roche, 2001) offer func- 
tional analytic alternatives to cognitive theoriz- 
ing in the form of derived relational responding. 

Using matching-to-sample (MTS) proce- 
dures, researchers train participants to select a 
particular stimulus, B, from an array of com- 
parisons in the presence of A. Similarly, C is 
trained as the correct selection in the presence 
of B. Following such an experimental history, 
participants are likely to select A from an array 
of alternatives when B is displayed (symmetry), 
B from an array when C is displayed (sym- 
metry), A when C is displayed (transitivity), and 
C when A is displayed (equivalence). This series 
of derived performances confirms that partici- 
pants are able to relate the training stimuli as 
equivalent forms of the same objects or events. 

Stimulus equivalence strategies have provided 
a functional system for generating a wide 
range of complex behaviors in low-functioning 
humans (e.g., Sidman, 1977; Sidman & 
Cresson, 1973; Trace, Cuvo, & Criswell, 
1977). In one early illustration, Mackey and 
Sidman (1984) taught students with mental 
retardation to identify colors verbally and to 
assemble the names of colors with alphabetic 
characters. Then, students matched these 
printed words to colors and the colors to their 
respective printed words. This MTS procedure 
allowed students to verbally identify the names 
of the printed words. 

Interestingly, there have been very few 
applied studies on stimulus relations that 
address the learning of mathematical operations. 
One exception is Lynch and Cuvo (1995), who 
developed a protocol wherein low-performing 
fifth- and sixth-grade students were trained to 


match fraction ratios (A) to their graph or 
pictorial representation (B). Then, participants 
were trained to match the graphs or pictures (B) 
to their matching decimal values (C). Upon 
emergence of equivalence, participants’ general- 
ization to other, untrained fraction-to-decimal 
relations was assessed. Although generalization 
was demonstrated for several students, this study 
limited its scope to fractions and their decimal 
equivalents; application to more advanced 
mathematical concepts has not been explored. 
One item of note is that Lynch and Cuvo’s 
study employed a computer voice synthesizer to 
provide a pretraining procedure that instructed 
participants regarding response contingencies; 
however, the participants never received infor- 
mation on rules regarding the trained mathe- 
matical relations, either from the voice 
synthesizer or the experimenter. Participants’ 
correct MTS responses resulted in a computer- 
generated affirmation of accuracy in the form of 
“yes.” However, Lynch and Cuvo reported that 
generalization might have been enhanced by 
“teaching students to develop general rules 
during training and posttest trials” (p. 125). 

In a somewhat similar procedure, Leader 
and Barnes-Holmes (2001) trained fraction- 
decimal relations to 24 5-year-old preschool 
children using fractions, decimals, and partially 
shaded pictures. Prior to MTS, this study 
employed a two-part training system. During 
demonstration trials, participants were told and 
shown how to point to correct comparison 
stimuli (e.g., circles divided into shaded quad- 
rants) when samples were presented in the form 
of fractions. During subsequent no-help trials, 
the experimenter presented samples and the 
participants independently pointed to compar- 
ison stimuli. Reinforcement was provided, 
contingent on correct identification of compar- 
ison stimuli, and inaccurate responses resulted 
in immediate verbal corrections. MTS test trials 
were conducted in the same format as no-help 
trials; however, no systematic consequences 
were provided. Following tests for equivalence, 
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follow-up experiments confirmed that these 
preschool children were better able to generalize 
their performances to more complex pictorial 
representations of fractions (in the form of a 
square composed of 16 shaded or nonshaded 
segments) when generalization tests were admin- 
istered according to a progression of increas- 
ingly novel comparisons. Here, as in the Lynch 
and Cuvo (1993) study, the relations among 
stimuli were expressed using the equivalence 
paradigm, and although instructions detailed 
how participants should point to particular 
objects under various conditions, verbal rules 
were not employed to train specific stimulus 
relations. 

The potential for humans to demonstrate 
other types of stimulus relations has been 
explored (Steele & Hayes, 1991), and several 
types of relational responses are described by 
RFT (see Hayes et al„ 2001, for a discussion). 
From an RFT perspective, responding in 
accordance with stimulus relations includes 
the properties of mutual entailment, combina- 
torial entailment, and the transformation of 
function. With respect to the property of 
mutual entailment, if a given stimulus is related 
to another, such that Stimulus A is the same as 
Stimulus B, then the derived relation B same as 
A is mutually entailed. This property operates 
in a manner consistent with symmetry (Sidman, 
1986). However, if A is greater than B, then the 
relation B less than A is mutually entailed. In 
another form of relations, if A is the opposite of 
B, then B opposite of A is mutually entailed. At 
the same time, if B is the opposite of C, then 
C opposite of B is mutually entailed. Given all 
of the above, the relation C same as A and A 
same as C is derived and described by RFT as 
combinatorial entailment (O’Hora, Roche, 
Barnes-Holmes, & Smeets, 2002). 

Behavioral researchers are now exploring 
innovative methodologies to investigate stimu- 
lus relations pertaining to sameness, opposite, 
more than, and less than (e.g., Dymond & 
Barnes, 1995); opposite and same (e.g., 


Dymond & Barnes, 1996); temporal, spatial, 
and deictic (e.g., Barnes & Roche, 1997); and 
opposition, distinction, analogical, and meta- 
phorical (e.g., Stewart, Barnes-Holmes, Hayes, 
& Lipkens, 2001; Stewart, Barnes-Holmes, & 
Roche, in press; Stewart, Barnes-Holmes, 
Roche, & Smeets, 2001). Even before most of 
these controversial studies addressed the poten- 
tial value of new approaches to multiple stimu- 
lus relations (see Galizio, 2003, for a discus- 
sion), Hayes and Barnes (1997) argued for the 
development of more basic and applied research 
to explore verbal learning and problem-solving 
applications (see Barnes-Holmes & Hayes, 
2003, for a response to Galizio). In our view, 
these studies do not contradict stimulus equiva- 
lence. However, RFT tries to identify additional 
variables that may interact with the emergence 
of new types of derived stimulus relations. 

In this vein, we were interested in determin- 
ing if RFT preparations might have practical 
problem-solving applications for instruction of 
particular types of mathematical relations. For 
example, several fundamental formulas (e.g., 
linear, square root functions, logarithmic, 
exponential, sine, tangent, etc.) have graphical 
functions, such that each type of formula 
operates on the coordinate axis in a way that 
is analogous to other types of mathematical 
functions. Very much like the arbitrary stimuli 
used in computer-interactive MTS procedures, 
for individuals with limited mathematical his- 
tories, formulas and their graphical representa- 
tions may seem completely ambiguous and 
arbitrary. However, an individual who has 
learned the relations between particular types 
of formulas and the graphs of these functions 
may be well positioned to identify new relations 
insofar as different types of formulas generate 
graphs (as well as other types of formulas) that 
are analogous. 

Most RFT, as well as stimulus equivalence, 
researchers have refrained from appealing to 
rule governance to explain the formation of 
stimulus relations or even to develop training 
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procedures that employ complex instructions 
(see Hayes, Barnes-Holmes, & Roche, 2003, for 
a discussion). In computerized procedures (e.g., 
Stewart, Barnes-Holmes, Roche, & Smeets, 
2002), the sample stimuli often appear in one 
sector of the screen while comparison stimuli 
are contrasted in separate partitions of the 
display. Participants click on what they believe 
to be the correct comparison stimulus and 
receive accuracy feedback from the computer. 
Although this is an elegant system for studying 
arbitrary stimulus relations (cf. Horne & Lowe, 
1996), it is not, nor was it intended to be, 
particularly efficient in teaching complex verbal 
relations in an academic setting. And, practi- 
cally speaking, mathematics instructors (or 
designers of computer-assisted mathematics 
software) would find it untenable to ask their 
students or software users to become skilled at 
complex mathematical operations without 
introducing them to precise rules for solving 
the problems in question (Ninness, McCuller, 
& Ozenne, 2000). Nevertheless, we believe 
that computer-interactive MTS platforms with 
instructional applications might incorporate 
functional analytic strategies based on stimulus 
equivalence and RFT protocols. 

In a recent study by Ninness et al. (in press), 
participants demonstrated mutual entailment 
on formula-to-graph and graph-to-formula rela- 
tions and were assessed for 36 novel variations 
of the original training formulas and their 
graphical representations. Many of the partici- 
pants demonstrated perfect or near-perfect per- 
formance on all novel relations. Nevertheless, 
other participants made several errors during 
the first assessment of novel formula-to-graph 
relations. Error patterns were classified with the 
help of a self-organizing map (Kohonen, 2001), 
and a new training protocol was developed to 
remediate the dominant error patterns exhibited 
during the first experiment. Following training 
on the revised protocols, all participants 
demonstrated a substantial reduction in errors 
relative to their earlier performance. 


The present investigation continues to focus 
on the transformation-of-function graphs (not 
transformation of stimulus functions, as dis- 
cussed in RFT); however, we have attempted to 
expand our training protocols. In reviewing 
strategies for training formula-to-graph rela- 
tions, it became apparent that most textbooks 
and software systems do not incorporate 
explanations in conjunction with direct practice 
in addressing formula-to-graph relations (cf. 
Zaskis, Liljedahl, & Gadowsky, 2003). This 
seems to be particularly true when the standard 
forms of formulas contain — 1 coefficients in 
the leading x variable [e.g., y — log(— x — 4)]. 
The present study extends our previous research 
in this area by employing a brief lecture 
followed by a computer-interactive program to 
train the relations between standard formulas to 
more easily understood factored formulas and 
factored formulas to their graphical representa- 
tions. The lecture was followed by computer- 
interactive training and MTS procedures that 
assessed mutual entailment and combinatorial 
entailment. Subsequently, participants were 
probed over an array of novel and complex 
formula-to-graph relations. 

It is worth noting that the formulas 
employed as sample or comparison elements 
were not unique, and any number of rearranged 
formulas might have generated the same graphs. 
Participants were not trained to relate a specific 
formula and graph as unitary stimuli; rather, 
they were trained to identify relations contained 
within the samples and comparisons. That is, 
we attempted to train participants to identify 
the relations contained within the formulas to 
the relations contained within other formulas 
and graph analogues. RFT, with its emphasis on 
conceptual and empirical analyses of identifying 
relations, seemed most appropriate for this type 
of experimental preparation. In training A-B, 
standard formulas (highest power variables 
first and constants last) served as samples and 
factored formulas served as comparisons. In 
training B-C, factored formulas served as 
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samples and graphs served as comparisons. 
Combinatorial entailment testing assessed 
participants’ understanding of the relations 
between the graphical versions of the formulas 
(C) and the original standard formulas (A) as 
well as C-A relations. Following demonstration 
of combinatorial entailment, participants re- 
sponded to 40 novel standard and factored 
formulas relative to their respective graphed 
functions. 

METHOD 

Participants and Setting 

Eleven participants, ranging in age from 15 
to 37 years, were recruited among university 
students and employees from a local hospital 
rehabilitation facility and their family members. 
Following informed consent and a pretest 
to determine level of familiarity with various 
mathematical functions, individuals who 
demonstrated any familiarity with algebraic or 
trigonometric transformation of functions were 
excluded from the experiment. Specifically, any 
potential participant who answered more than 2 
of 15 pretest items correctly was eliminated 
from the participant pool. 

One participant required a specialized train- 
ing format. Participant 6 was a 15-year-old boy 
who had previously been diagnosed as dyslexic. 
During training, he was unable to read most of 
the directions on the computer screen for the 
formula-to-graph modules, so the experimenter 
read the directions on the screen for the A-B 
and B-C modules to him. However, no other 
form of assistance was provided during any part 
of the training session, and the participant 
performed all items on the assessment of novel 
relations independently. 

Undergraduate and graduate student partici- 
pants earned 3 extra points toward their final 
class examination grade. Hospital employees 
and their family members were recruited 
through personal contacts and received financial 
reimbursement ($10) for their participation. In 
addition, all participants were paid 10 cents for 


every correct response during the assessment-of- 
novel-relations phase of the study (maximum of 
$4). After completion of the experiment, all 
participants were debriefed and compensated. 
The experimental sessions were carried out in 
empty classrooms of the university campus or 
offices of the hospital rehabilitation facility. 
These settings were arranged to preclude dis- 
tractions or interruptions, and they remained 
free of noise throughout the study. 

Apparatus and software. All software for this 
study was developed on a Dell Dimension® 
4400 computer (Pentium 4 1.8-GHz processor 
with 768 MB RAM). The mathematical 
instructional and MTS modules were written 
in Microsoft® Visual Basic 6 for IBM PC- 
compatible machines by the first author. The 
software provided math instructional tutorials 
and displayed graphs, and it assessed and 
recorded the speed and accuracy of user 
performance during all phases of the study. In 
addition, the first author modified the self- 
organizing map algorithm in C++ to categorize 
error patterns based on Kohonen’s logic for 
these algorithms (see Kohonen, 2001, for a 
discussion). The experimental procedures were 
conducted on a Hewlett-Packard Pavilion® 
ze5170 (Pentium 4 2-GHz processor with 512 
MB RAM) with an attached infrared mouse. A 
microphone sat adjacent to the computer, and it 
was conspicuously attached to the side port of 
the computer. 

Design and Procedure 

Following informed consent, participants 
who demonstrated no familiarity with algebraic 
and trigonometric functions relative to vertical 
and horizontal shifts were asked to continue 
with the experiment. Following a brief pre- 
training presentation, participants were escorted 
to their respective computers in isolated rooms 
and computer-interactive training was initiated. 
Procedures included training and testing of A-B 
and B-C relations and the assessment of 
mutually entailed (B-A and C-B) and combi- 
natorially entailed (A-C and C-A) relations. The 
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program then assessed participants on 40 novel 
relations between formulas and the graphs of 
these functions (see Table 1). 

Stage 1: Pretraining presentation of basic 
mathematical relations. The presenter con- 
ducted pretraining lectures in small groups (2 
or 3 participants at a time) and included 
projections of formulas and graphs on an 
overhead screen above the lectern. The pre- 
senter read rules regarding basic mathe- 
matical operations directly from the overhead 
transparency and responded to questions from 
participants only insofar as they were directed at 
sample-to-comparison items (A-B or B-C). At 
no time were the relations of B-A, C-B, A-C, 
and C-A displayed or discussed. 

Step 1: Provide A-B rules. In pretraining A-B 
rules, it was explained that a negative coefficient 
of x inside the standard form of a formula often 
complicated the graphing of a function. It was 
further explained that the standard form of the 
square root formula could be factored to 
remove any negative signs that precede the x 
variable. 

In describing this and all A-B relations, 
standard formulas (highest power variables 
first and constants last) served as illustrations 
of samples, and factored formulas served as 
comparisons. The participants were shown a 
basic square root function in its standard form 
and how it can be expressed when a — 1 


coefficient is factored out of the argument. In 
the pretraining and computer training, we used 
the words negative sign rather than — 1 
coefficient and conducive rather than factored 
because most of our participants had 
no familiarity with, or misleading interpre- 
tations of, the latter terms. 

Step 2: Provide B-C rules. Because many of 
our participants were unacquainted with the 
rectangular coordinate system, the presenter 
explained that the horizontal number line is 
called the x axis and that the vertical number 
line is referred to as the y axis. Further, it was 
explained that various types of formulas were 
called functions and could be used to generate 
graphs on the coordinate axes. In describing 
functions for B-C relations, factored formulas 
served as samples and graphs served as compar- 
isons. Using an overhead transparency, the 
presenter read rules explaining that a negative 
sign inside the radical reflects the graph over in 
(about) the y axis and that a negative sign 
outside the radical or the parentheses reflects the 
graph down in (about) the x axis. Similarly, the 
presenter explained that a constant value added 
or subtracted outside the radical or the 
parentheses moves the graph up or down the y 
axis in the same direction indicated by the sign. 
The overhead transparency was identical to a 
screen later used by the computer-interactive 
program that followed this lecture. 


Table 1 

Sequence of Training and Assessment Conditions 


Stage 1. Pretraining presentation on basic relations 

Provide A-B rules Provide A-B exemplar with formulas 

Provide B-C rules Provide B-C exemplar with formula and graph 


Stage 2. Computer-interactive training and testing of complex relations 


Trained relations 

Mutual entailment 

Combinatorial entailment 

Al-Bl 

Bl-Al 

Al-Cl 

Bl-Cl 

Cl-Bl 

Cl-Al 

A2-B2 

B2-A2 

A2-C2 

B2-C2 

C2-B2 

C2-A2 

A3-B3 

B3-A3 

A3-C3 

B3-C3 

C3-B3 

C3-A3 


Novel relations 

[e.g y = — sinf.r) + 6] 


A4-B4 

B4-C4 


B4-A4 

C4-B4 


A4-C4 

C4-A4 
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Step 3: Provide A-B exemplar. In this step, 
participants were shown an example of an 
A-B test screen, where A was represented as a 
standard square root formula and B as one of six 
factored square root formulas or comparisons. 
The presenter explained that a very similar 
type of selection format would be employed in 
the forthcoming computer-interactive training 
session and used a pencil to tap the letter of 
the correct comparison item displayed on the 
overhead. 

Step 4: Provide B-C exemplar. The partici- 
pants were shown an example of a B-C test 
screen, where B was the factored formula and C 
was one of six possible graphical representations 
of that function. Again, the presenter used a 
pencil to tap the letter of the correct comparison 
item displayed on the overhead. (These A-B and 
B-C examples were not arranged in the same 
positions as those employed during the actual 
computer-interactive training used in Stage 2.) 

All participants were exposed to the same 
pretraining presentation regarding the funda- 
mentals of the rectangular coordinate system 
and the relation between the square root 
formula and its graphical representation. The 
entire presentation required approximately 
15 min. Following the small-group presenta- 
tions, participants were escorted to their respec- 
tive computers, each of which was located in a 
separate room. Before allowing the participants 
to initiate the computer-interactive training 
and assessment program, the presenter demon- 
strated a point- and-click response on a sample 
screen and verified that the participants could 
perform the same response independently. 

Stage 2: Computer-interactive training and 
testing of mathematical relations. During com- 
puter-interactive training in Stage 2, our 
program emphasized giving participants visual 
displays and audio output in the form of 
mathematical rules and having participants 
repeat these rules immediately by reading them 
into a microphone. In conjunction with the rule 
rehearsal, participants engaged in repeated 


practice of the behaviors specified by the rules. 
After training and assessing formula-to-formula 
(A-B) relations and formula-to-graph (B-C) 
relations, participants were assessed for 
mutually entailed formula-to-formula (B-A) 
relations and graph-to-formula (C-B) relations. 
Subsequently, the combinatorially entailed rela- 
tions between the graphical versions of the 
formulas and the original standard formulas (C- 
A), and the standard form of the formulas and 
their respective graphical representations (A-C), 
were assessed. 

Step 1: Train A-B relations, test A-B. 
Computer-interactive training of A-B relations 
included several of the details described to the 
participants during the pretraining presentation. 
The computer provided prerecorded audio 
output and displayed mathematical rules on 
each screen. Following the audio expression 
of the rule, the participants read the same 
computer-posted rules into the microphone. 
For example, participants heard and recited 
rules indicating that the standard form of the 
square root formula could be factored to 
remove any negative signs that precede the x 
variable (see Figure 1, top). After participants 
heard and recited these rules twice, clicking 
“next” moved to them to a screen that assessed 
A-B performance (as shown in Figure 1, 
bottom). 

Step 2: Train B-C relations, test B-C. The 
same type of audio output, display, and rule- 
recitation format was employed to train B-C, 
test B-C performance. Training B-C relations 
included having the participants hear and recite 
rules regarding reflections and vertical and 
horizontal shifts associated with various func- 
tions on the coordinate system. The screen 
displayed solid blue lines to represent the basic 
square root function and dashed red lines to 
show various transformations of the square root 
functions. As illustrated in Figure 2 (top), these 
rules indicated such mathematical relations as, 
“Negative sign inside the radical is reflected 
over in the y axis. A positive constant inside the 
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Moving from the standard form of the formula to the 
most conducive form of the formula, the conducive form 
always factors out any negative signs that may precede x. 


y = '/-.v - 2 



y “ 4 m X - 2 


m r 

y = 4x + 2 


to 

y = s}x~ 2 

y = 'i-v(-2) 

m 

<N ( 

A 

n 

5 ^ 

y = \x + 2 


Figure 1 . Two training screens are displayed. The top screen shows standard and factored (conducive) forms of the 
square root formula when a negative coefficient precedes the x variable and a subtraction operation (—2) is provided 
inside the argument. Below, an A-B test shows the correct response to the trained relations circled. 


radical or the parentheses moves the function in 
the opposite direction along the x axis.” (We 
used the words over in rather than the 
traditional about in response to pilot testing 
that suggested this phrasing seemed to be easier 
for many participants to follow.) After partici- 
pants heard and recited these rules twice, 
clicking “next” moved them to a screen that 
assessed B-C performance, as illustrated on the 
lower half of Figure 2. Note that although the 
assessment screens varied the specific values of 
the constants within the formulas, the correct 
answers were always obtainable by behaving in 
accordance with the rules displayed on the 
previous screens. 


Step 3: Test B-A, test C-B, test A-C, and test 
C-A. This step examined the mutually entailed 
B-A and C-B relations as well as the combina- 
torially entailed relations between the graphical 
versions of the formulas and the original stan- 
dard formulas (C-A) (Figure 3, top). Likewise, 
the combinatorial relations between the stan- 
dard forms of the formulas and their respective 
graphical representations (A-C) were assessed 
(Figure 3, bottom). 

In total, the program aimed to develop four 
three-member classes of relations made up of 
formula-to-formula and formula-to-graph rela- 
tions. All mathematical rules, formulas, and 
graphical representations for horizontal and 
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Negative sign inside the radical is reflected over in the v-axi n 

A positive constant inside the radical or the parentheses moves the 
function in the opposite direction along the .r-am 





Figure 2. On top, a formula-to-graph illustration and mathematical rules pertaining to square root function transforma- 
tions are displayed. All mathematical rules, formulas, and graphical representations for horizontal and vertical shifts as well as 
those describing reflections in the x axis and y axis were presented in a similar format. A line drawn from the formula to the 
graph indicates the reason for the transformed function. At the bottom of the figure, a B-C test screen shows the correct response 
to the trained relation. The solid lines indicate the graph of a basic square root function ( y = yfx). The dashed lines indicate 
the possible transformation of the graph according to the sample formula. In each case, the correct response has been circled. 
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y = 




? ■ 
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Figure 3. On top, a test for the combinatorial relation between the graphical version of a formula and the standard formula 
(C-A) is displayed. Beneath, the combinatorial relation between the standard forms of the formula and its graphical 
representation (A-C) is assessed. Solid lines indicate the basic square root function ( y = y/x), and the dashed line indicates 
the possible transformation of the graphs according to the sample formula. In each case, the correct response has been circled. 
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vertical shifts as well as for reflections in the x 
axis and y axis were presented according to this 
format. Specifically, the program provided 
training and testing on two versions of the 
square root function and two versions of the 
common logarithmic function. Figures 4 and 5 
illustrate the trained, mutually entailed, and 
combinatorially entailed relations. Figure 4 
specifies the elements of basic square root 
function as Al, Bl, and Cl and a more 
complex square root function as A2, B2, and 
C2. Figure 5 shows the training and testing 
form for the log functions A3, B3, and C3 


Al 



Bl 

y - V-(.v + 2) 


followed by a more complex log function of A4, 
B4, and C4. After each trained (A-B and B-C) 
relation, the program assessed for mutually 
entailed (B-A and C-B) relations as well as 
combinatorially entailed (C-A and A-C) rela- 
tions. If a participant failed to produce a correct 
response during any MTS tests, they were 
reexposed to the entire relational training and 
testing cycle, beginning with the first (Al-Bl) 
training screen. 

Correction procedures and mastery crite- 
ria. Our mastery criterion required partici- 
pants to complete one errorless sequence of all 


2 


\\ 



y 


v = 



■V- (x + 4 ) + 6 


A2 

6 - \j~x - 4 

\\ 

v' 

V' 

C2 



Figure 4. The top section shows the trained and tested elements of basic square root function Al, Bl, and Cl. 
Below, a more complex square root function A2, B2, and C2 is displayed. 
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AS 


y = l°g(-.v - -0 



B3 

j? — log(-(a* + 4)) 


\\ 

x'- 



A4 

v = 'S+log(-.\'- 4) 



B4 


y — log(-(.v + 4)) -8 


\\ 

\ N 

A 

A 

C4 



Figure 5. The top screen shows training and testing of the log functions A3, B3, and C3, followed a more complex 
log function of A4, B4, and C4 at the bottom. Note that in the graphical representation of our logarithmic functions, we 
do not show/(x) continuing — oo. 


four classes of relations, each consisting of six 
MTS problems (A-B, B-C, B-A, C-B, A-C, and 
C-A). Mastery was comprised of 24 consecutive 
correct MTS responses. A participant who made 
an error during the assessment of trained, 
mutually entailed, or combinatorially entailed 
relations was cycled back to the beginning of 
the program and was reexposed to the entire 
training and testing sequence. Here, the 
program randomized all comparison elements 
before initiating the new training cycle. In the 
event that a participant required more than five 


training cycles, the program terminated, and the 
participant was paid, debriefed, and excused 
from the experiment. 

Assessment of novel relations. Following 
demonstrations of mutual entailment and 
combinatorial entailment on both versions of 
the square root functions and both versions of 
the common logarithmic functions, the pro- 
gram assessed participants on novel relations 
between formulas and their graphically repre- 
sented functions. Novel algebraic and trigono- 
metric functions were composed of 40 complex 
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y — - (x-+- 4) 



v = - sin(.v) - 6 



Figure 6. Two of the 40 tests of novel formula-to- 
graph relations are shown. The solid lines represent the 
basic squaring function (y = x 2 ), and the dashed lines 
indicate the possible transformation when the formula 
becomes more complex. On top, a participant who 


variations of the originally trained mathematical 
relations. Each test item required participants to 
match a new formula with a graph from an 
array of six that had not been employed during 
any of the training and assessment screens. (The 
comparison graphs were all of the same type of 
mathematical function; e.g., a logarithmic 
formula was compared only against variations 
of logarithmic graphs.) No form of accuracy 
feedback or recycled training was provided 
during any of the tests for novel relations. 
This task entailed identifying graphs that 
represented formulas with new combinations 
of positive and negative constants preceding and 
following various square root and logarithmic 
functions. Moreover, testing novel relations 
included multiple combinations of reflections 
and vertical and horizontal shifts for complex 
exponential, square, cubic, tangent, and sine 
functions. 

Figure 6 shows two of the 40 tests for novel 
relations. When presented with a formula for 
a squaring function that included a leading 
negative one coefficient preceding the argument 
and positive constants within the argument, a 
participant who answered correctly would select 
D as the graphed reflection and accompanying 
shift along the x axis. If shown a formula of a 
sine function in which a — 1 coefficient pre- 
ceded the argument and a positive constant 
was placed after the argument, a participant 
who identified the transformation of function 
correctly would select C. 

As additional probes of novel relations, three 
of the novel formulas contained correct but 
somewhat misleading constant values as prefix 

identified a novel variation of formulas that include a 
negative one coefficient in front of the argument and a 
positive constant within the argument would select D. 
Beneath, the solid lines represent the basic sine function 
[y = sin(x)], and the dashed lines indicate the possible 
transformation when the formula becomes more complex. 
A participant who identified the transformation that 
occurs with a negative one coefficient in front of the 
argument and a positive constant is placed after of the 
argument selected C as the correct response. 
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and postfix operations (—1 with +1 and —4 
with +4); specifically, participants were shown 
the following: y — — 1 + log (— x — 4) + 1 and 
y = —4 + y/—(x + 4) + 4. The third formula 
also contained redundant negative coefficients 
in the prefix: y — — (x + 4) . Again, all three 
probes were algebraically correct and consistent 
with an assessment of novel mathematical 
relations, but they were unlikely to be encoun- 
tered as actual formulas within most texts or 
software systems. All of the other sample 
formulas in the assessment of novel relations 
were presented in their usual standard or 
factored forms. 

RESULTS 

Ten participants completed all training 
modules within twenty or fewer training 
screens. Voice recordings of these participants’ 
recitations of rules were audible and complete. 
Table 2 shows the number of exposures needed 
for the 10 participants to complete the trained 
A-B and B-C relations. Most participants mas- 
tered the computer-interactive training se- 
quence in a relatively small number of exposures. 

Errors for each participant are displayed in 
Figure 7. The number of MTS errors on each 
tested relation is represented with shaded digits 
1 or 2, corresponding to the number of errors 
made. Open blocks of zeros represent the 
absence of an error during the assessment of a 
given relation. Figure 7 shows that 7 of the 10 
participants made one or more errors in 
deriving A-C or C-A combinatorial relations. 


Participants 1 and 1 0 made errors in identifying 
Cl-Al when the comparison formula included 
a leading negative coefficient of x in the argu- 
ment in the standard form of the square root 
function (Figure 4). Participants 5, 7, and 9 
failed to derive the combinatorial relation 
C2-A2. This square root formula involved a 
constant (6) and a negative coefficient in front 
of the argument. It also included a negative 
coefficient of x and a negative constant inside of 
the argument. Participant 7 made this same 
error in the form of A2-C2 when he attempted 
to identify the graph that matched this formula. 
Accordingly, he was cycled back to the begin- 
ning of the program on two separate occasions 
and received three training exposures (Table 2). 
Participants 2 and 3 made errors in deriving 
C3-A3 combinatorial relations when the loga- 
rithmic formula contained a negative x followed 
by a negative constant inside the argument 
(refer to Figure 5). Participant 3 made the same 
incorrect response on two consecutive occasions 
and was recycled through the program accord- 
ingly. Participants 4, 6, and 8 made no errors 
during the assessment of trained or derived 
relations. 

Only 2 participants made errors with regard 
to the property of mutual entailment. During 
her second cycle through training and testing, 
Participant 5 failed to derive C3-B3 when the 
sample graph reflected about the y axis and 
shifted in the negative direction along the x axis. 
The correct comparison formula was a negative 
logarithmic function in its factored form (refer 


Table 2 

Number of Training Exposures Required to Attain Mastery 
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Figure 7. Errors for each participant are displayed. The number of MTS errors on each tested relation is represented 
with shaded digits 1 or 2. A 0 represents the absence of an error on a given test. 


to Figure 5). Participant 10 made a similar error 
during her second cycle in deriving C4-B4 
when the sample graph reflected about the y 
axis, shifted in the negative direction along the x 
axis, and shifted in the negative direction along 
the y axis. The correct comparison formula was 
in its factored form of the logarithmic formula. 
As shown in Table 2, both participants were 
cycled back to the beginning of training when 
these errors occurred. 

The top of Figure 8 shows the standard forms 
of the original four formulas (Al, A2, A3, and 
A4) in their standard forms employed during 
training. These four formulas (in the center of 
the illustration) served as the foundation for 
developing the trained and derived relations. 
The periphery shows the 40 formulas that were 
employed in probing novel formula-to-graph 
relations. The y subscripts identify the order in 
which these formulas appeared in the assess- 
ment sequence (e.g., y 1 4 — 4 (x 4) was the 14th 
item on the test of novel relations). 


The center of Figure 8 shows the pretest 
errors on 15 formula-to-graph test items. The 
density of pretest errors in this middle block 
confirms that our participants were unfamiliar 
with the formula-to-graph relations prior to 
training. 

The four bottom blocks show the same 
participants’ errors following MTS training, as 
classified and grouped by the self-organizing 
map. The first 15 items were the same as those 
used in the pretest but were novel in terms of 
the participants’ MTS training experience. 
The remaining 25 items (16 through 40) 
consisted of an additional 25 novel stimuli that 
were not used during any part of training or 
pretraining. 

Error Pattern 1 contains data for the 6 
participants who completed training with per- 
fect or near-perfect performance, obtaining at 
or above 92.5% accuracy in identifying novel 
algebraic and trigonometric formula-to-graph 
relations. With only a few exceptions, the small 
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Tests for Novel Relations 


Figure 8. The top shows the four standard functions (four formulas within the center oval) employed during training 
and assessment of stimulus relations. Formulas on the periphery were used as sample stimuli in testing novel formula-to- 
graph relations. The middle block shows the correct and incorrect responses on the pretest. Problem numbers are listed along 
the x axis for each of the 10 participants. Accurate responses contain the digit 0; errors are shaded blocks that contain 1. 
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number of errors was associated with not 
recognizing graphical representations when the 
formula contained redundant negative signs 
preceding the argument (j /29 in Figure 8 ) or 
when a negative one coefficient preceded a sine 
function (y 36 in Figure 8 ). Participant 6 
demonstrated the largest number of errors in 
this group. These included items that addressed 
a double negative coefficient, an exponential 
function that included a constant following the 
argument, and a negative sine function. 
Participants 7 and 10 misidentified only one 
of the novel formula-to-graph relations. 

Error Pattern 2 shows 2 participants who 
commonly misidentified formula-to-graph rela- 
tions of exponential and negative sine functions 
as well as errors associated with formulas 
containing double negative signs preceding the 
argument. Error Pattern 3 includes 1 partici- 
pant who made a series of incorrect responses 
that have much in common with those in Error 
Pattern 2. Errors for this participant were 
associated with several different types of 
formulas that contained negative or double 
negative signs preceding their arguments. This 
participant also misidentified a cubic function 
and a negative sine function. Error Pattern 4 
shows outcomes for the only participant who 
demonstrated a somewhat more diversified and 
scattered series of errors across a wider range of 
novel formula-to-graph relations. Participants 
averaged 38 min (range, 26 min to 57 min) to 
complete training, MTS testing, and probing of 
novel mathematical relations. 

There appears to be little correspondence 
between the total number of training cycles 
to reach mastery and the number of errors 
obtained during the assessment of novel rela- 
tions that followed. For example, Participant 5 
was exposed to a total of 1 8 training screens and 
made the largest number of errors during the 
assessment of novel relations. Participant 10 was 
exposed to a total of 18 training screens and 
made only two errors in the assessment of novel 
relations. 


DISCUSSION 

These results are consistent with our previous 
study (Ninness et ah, in press), in which 
participants demonstrated mutual entailment 
on formula-to-graph functions and subse- 
quently identified relations between new formu- 
las and their respective graphically displayed 
representations. The present results advance the 
previous work by employing MTS strategies 
that assess mutual entailment and combinatorial 
entailment and propagate a more extensive 
repertoire of formula-to-graph relations. Novel 
formula-to-graph relations included identifying 
graphical functions when formulas involved 
combinations of constants in new positions 
inside and outside function arguments, com- 
binations of positive and negative signs in the 
prefix and postfix operations, and several new 
and more detailed algebraic and trigonometric 
functions (sine, exponential, cubic, and tangent 
functions) that included new combinations of 
variables and constants inside and outside the 
arguments of these novel formulas. We are not 
suggesting that, following this brief training and 
MTS exercise, participants understood all of the 
mathematical dynamics of these functions; 
however, having learned to factor particular 
types of formulas and to identify particular 
graphical functions with a limited array of 
exemplars, participants demonstrated a general- 
ized ability to derive a wider range of more 
complex formula-to-graph relations. 

Although several participants had taken 
intermediate algebra, none of them had any 
familiarity with the procedures provided during 
training. Preexperimental inquiry and pretesting 
revealed that they had no understanding of 
mathematical functions addressing vertical and 
horizontal shifts or reflections. Moreover, they 
were unable to factor standard versions of for- 
mulas correctly. In fact, any potential partici- 
pant who answered more than 2 of 15 pretest 
items correctly was eliminated from the parti- 
cipant pool. As noted above, Participant 6 was 
a 15-year-old boy who had previously been 
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diagnosed as dyslexic. During training, he was 
unable to read most of the directions in the 
formula-to-graph modules, so the experimenter 
read the directions for the A-B and B-C 
modules to him. However, no other form of 
assistance was provided during any part of the 
session. This participant proceeded to get every 
B-A, C-B, A-C, and C-A comparison correct on 
the first trial throughout all test screens. The 
assessment of novel relations that followed did 
not require any reading of directions, and he 
performed this phase of the study indepen- 
dently. He missed only three of the 40 complex 
variations of the original formula-to-graph 
relations. 

Caveats 

Our interactive training software is not suf- 
ficiently developed for general dissemination. 
Nevertheless, the general approach described in 
this study might well be adapted to traditional 
modalities of instruction. Indeed, Lynch and 
Cuvo (1995) and Leader and Barnes-Holmes 
(2001) conducted a series of instructional pro- 
cedures that employed many of the steps de- 
scribed herein. In addition, we are developing a 
Web-based version of this project on our Web 
site at www.lcsdg.com/psychStats. Upon com- 
pletion, it will be freely available to interested 
users. 

This study did not attempt to differentiate 
the influence of the pretraining lecture from the 
effects of computer-interactive MTS training 
procedures that followed. Because most of our 
participants were unfamiliar with the x, y 
coordinate system, some minimal level of 
preparation was required for them to interact 
effectively with the computerized training 
program. Nonetheless, future research should 
attempt to isolate the entry level of skills critical 
to initiating training on such platforms. 

Four participants displayed five or more 
errors during tests of novel relations. The domi- 
nant error pattern among these participants was 
failure to recognize the graphs associated with 
exponential functions. This may not have been 


due to the inherent complexity of the function 
per se as much as the particular elements within 
these functions. That is, the four sample for- 
mulas that we employed in our assessment of 
novel exponential functions all included a base 
of 4 in conjunction with various forms of 
exponents of 4 (e.g., y = 4^ A+4) ). The simul- 
taneous use of 4 in the base and exponent 
within these novel formulas may have com- 
pounded the ambiguity of the expressions. The 
error patterns of 2 other participants showed 
most of the above errors plus several diversified 
and idiosyncratic errors. Future research should 
attempt to identify variables that interact with 
such error patterns. 

Tests for Novel Relations in Applied Settings 

Manipulating symbols to identify new rela- 
tions among variables is a ubiquitous character- 
istic of human behavior. Most conspicuously, 
problem solving in science and technology 
requires the abstraction and reorganization of 
complex stimuli into new relations among 
objects and events. Tests for novel relations 
allow participants to identify different and more 
complex sequences of stimulus relations not 
included in the training protocol. 

The mathematical relations reported by 
participants in the current study (formula to 
graph) are not specifically symmetrical. Show- 
ing participants novel formulas allowed them to 
identify a vast network of relations extending 
from the exemplars provided during training. It 
might be argued that if our participants were 
learning to form four three-member classes 
made up of formulas and graphs in the same 
way as they would in using completely arbitrary 
shapes or nonsense syllables, then we might 
expect less adequate performance in tests 
involving novel stimuli. However, one interpre- 
tation is that they were not relating formulas 
and graphs as unitary stimuli but were identi- 
fying the relations contained within the samples 
and comparisons. This view is consistent with 
findings from previous basic research investigat- 
ing relational frames pertaining to same as, 


FUNCTIONAL ANAL YTIC MA THEMA TICS 


19 


more than, and less than (e.g., Dymond & 
Barnes, 1995) as well as temporal, spatial, and 
deictic relations (e.g., Barnes & Roche, 1997). 
One advantage to this perspective is that it may 
lend itself to the development of better MTS 
training platforms and allow students to learn 
composite relations among more complex 
operations. 

As in Ninness et al. (in press), our outcomes 
suggest that error patterns displayed by partici- 
pants are not necessarily due to negligence, nor 
are they necessarily the result of insufficient 
practice. Debriefing revealed that many partici- 
pants made incorrect inferences regarding 
problem-solving strategies. In some of our pilot 
and earlier research in related areas (e.g., 
Ninness & Ninness, 1998, 1999), we found 
that several incorrect problem-solving strategies 
produced correct answers intermittently and 
thus perpetuated intermittent errors as well as 
occasional accurate responses throughout the 
experimental sessions. Given these fairly com- 
mon circumstances in working with many types 
of math problems (Ashlock, 2002), it is hardly 
surprising that some students become confused 
and exasperated while trying to learn mathe- 
matical relations. Our present adaptation of the 
self-organizing map operates somewhat like a 
descriptive analysis insofar as it classifies error 
patterns across various types of problems within 
the study. These error patterns may be useful in 
refining the next generation of computer- 
interactive training platforms. Although it is 
true that a rigorous visual inspection of errors 
would eventually produce a similar pattern of 
correct and incorrect responses within and 
between participants, the self-organizing map 
performs this operation almost instantly while 
allowing the user to calibrate the software at 
specific levels of precision. 

In a previous study (Ninness et ah, in press), 
10 of 15 participants required supplemental 
training to derive nearly all relations successfully. 
Having access to a pattern of errors allowed us to 
intervene more efficiently with a second level of 


MTS procedures. Error patterns classified by the 
self-organizing map allowed us to provide 
specific remediations regarding formula-to- 
graph relations that participants found particu- 
larly challenging. However, in the current 
version of our experimental architecture, based 
on previously identified error patterns, we 
provided a more intensive training platform 
than existed in any of our previous attempts. In 
this study, MTS procedures repeatedly assessed 
participants for mutual entailment and combi- 
natorial entailment on two versions of a square 
root function and two versions of a log function 
before allowing them to be assessed on novel 
relations. Following training, 8 of 10 partici- 
pants in this study obtained at or above 85% 
accuracy on tests of novel relations, and 6 of 
these 10 obtained 92.5% or better. Following 
training, participants were no more likely to 
make errors associated with novel variations of 
A-C [e.g., y — ( — x — 4) 3 ] than they were to 
make errors that were novel variations of B-C 
[e.g., y — (x — 4) 3 ]. This result suggests that 
linear training procedures that incorporate tests 
for mutual entailment may be fairly efficient in 
terms of generating expanded mathematical 
repertoires. 

Future MTS procedures might focus on 
mutually entailed and combinatorially entailed 
mathematical relations that emerge from more 
robust and sophisticated computer-interactive 
MTS platforms. Such investigations might 
survey the combination of MTS preparations 
in conjunction with feedback from the self- 
organizing map to identify error patterns within 
and between participants. At the same time, 
training that focuses on the combinatorially 
entailed properties of more complex trigono- 
metric and calculus functions and their graphi- 
cal representations might prove beneficial for 
applied and basic research purposes. 

Similarly, procedures that further explore 
the rudiments of relational skills in algebra 
still are needed. For example, a quadratic func- 
tion expressed as y — x z — 6x entails factoring 
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(completing the square) to obtain y — (x — 3) 2 

— 9. This factored version of the formula may 
allow a student who has learned to identify 
formula-to-graph relations by way of the above 
experimental exercises to identify y — {x — 3) 2 

— 9 as a graph that shifts horizontally by three 
units (to the right) and vertically nine units. 
Pilot research in our laboratory suggests that 
several individuals who had difficulty identify- 
ing graphs of quadratic functions were able to 
learn these relations more efficiently following 
the type of MTS training used in this study. 
Moreover, learning to identify these relations 
appears to be generative, which may be useful in 
training higher levels of mathematical opera- 
tions. In calculus, for example, it may be 
beneficial to consider an intricate function as 
the sum, difference, product, or quotient of a 
less complex network. 

Research that moves beyond the present 
study to more sophisticated architectures and 
more extensive MTS procedures might provide 
a much wider range of combinatorially entailed 
formula-to-graph and graph-to-formula rela- 
tions. Of course, as in the teaching of phonics 
via direct-instruction procedures, there are 
exceptions to the rules of nonarbitrarily derived 
mathematical relations. For example, there are 
numerous odd functions in which negative 
coefficients positioned outside or inside the 
argument generate identical graphs. As one 
illustration, the function y = — sin(x) is gra- 
phically indistinguishable from y — sin(— x). 
Nevertheless, odd (and even) trigonometric 
functions are explicitly governed by shifts and 
reflections that are variations of the basic stra- 
tegies employed in this experiment. From our 
perspective, most basic and advanced formula- 
to-graph relations are well coordinated within a 
hierarchical network of relational frames that 
could be addressed within a more comprehen- 
sive computer-interactive tutorial that empha- 
sizes MTS training. One functional alternative 
to waiting for students to construct improved 
schemas of mathematical understanding might 


include the development of improved MTS 

protocols. 
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STUDY QUESTIONS 


1. Given Stimuli A, B, and C, of which A-B and B-C are directly trained in matching relations, what 
stimulus relations emerge according to (a) a stimulus equivalence account and (b) a relational frame 
theory account, and what are these relations called? 

2. What were the three mathematical forms (A, B, C) that were of interest in the current study? 

3. Briefly describe the training and testing of mathematical relations (Stage 2). 

4. What was the mastery criterion, and what consequence was arranged for incorrect responding? 

5. Briefly describe the assessment of novel mathematical relations. 

6. Describe the data depicted in Figure 7. What do these results show with respect to effects of 
training? 

7. The authors stated that participants were unfamiliar with novel formula-to-graph relations prior to 
training. What data support this statement? 


8. How might the data on the bottom panel of Figure 8 be used in a prescriptive manner? 


Questions prepared by Pamela Neidert and Erin Camp, University of Florida 


